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Abstract 

The emergence of increasingly sophisticated communication technologies and the media-rich 
extensions of the World Wide Web have prompted universities to use alternatives to the tradi¬ 
tional classroom teaching and learning methods. This demand for alternative delivery me¬ 
thods has led to the development of a wide range of eLearning techniques. Nonetheless, skep¬ 
ticism towards delivery methods as a "means of communication" is still common. This conflict 
has influenced research on the subject. Many studies have examined the effectiveness of 
eLearning (also called "technology-enabled learning" and "online learning") but very few of the 
results can be generalized to Information Technology/Information Systems curricula. The cur¬ 
rent study is an extension of a previous study conducted by the authors that examined the 
perceived effectiveness of online learning courses in a Computer and Information Systems 
curricula. The current study probes deeper into various online learning formats and into the 
students who enroll in the courses. Specifically, this new study looks at which non-traditional 
method of course content delivery (i.e., online, partially online, on-ground with online supple¬ 
ments) is most effective for Information Technology/Information Systems courses, as well as 
which instructional method of course content delivery provides the best learning for each 
grouping of Information Technology/Information Systems subject areas. Finally, this study 
attempts to profile groups of students who prefer an online format to a traditional on-ground 
format 

Keywords: Online Learning, eLearning, Web-Based Learning, Technology-Enabled Learning, 
IT/IS curricula 


1. INTRODUCTION 

The emergence of increasingly sophisticated 
communication technologies and the media- 
rich extensions of the Internet have prompt¬ 
ed universities to use alternatives to the tra¬ 
ditional classroom teaching and learning me¬ 
thods. This demand for alternative delivery 


methods has allowed new developments in 
the way instructors transfer course content 
to their students. These new developments 
have resulted in the growth of a new para¬ 
digm in pedagogy: technology-enabled 
learning environments. Numerous terms 
have been used to describe this phenome- 
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non, including Computer-Based Learning, 
Web-Based Learning, Technology-Supported 
Learning, eLearning, Distance Learning, and 
Online Learning. For purposes of this re¬ 
search, the term online learning will be used 
to describe any higher educational course 
offering that uses technology (i.e., software, 
electronic mail, and/or the Internet) to de¬ 
liver all or part of course content. 

The proliferation of online learning within 
higher education has stimulated the ongoing 
debate as to which delivery method is more 
effective: online learning or traditional class¬ 
room learning (Noble, 2002, O'Malley, 
1999). The present research looks at which 
non-traditional method of course content 
delivery (i.e., online, partially online, on¬ 
ground with online supplements) is most 
effective for Information Technology/ Infor¬ 
mation Systems (IT/IS) courses as well as 
which instructional method of course content 
delivery provides the best learning for each 
grouping of IT/IS subject areas. Also, this 
study attempts to profile groups of students 
who prefer an online format to a traditional 
on-ground format. 

2. BACKGROUND 

Online learning has been cited as having 
many distinct advantages over traditional 
classroom learning. Those advantages in¬ 
clude 1) reduced time to proficiency, 2) low¬ 
er logistical costs, 3) added convenience, 4) 
personalized curricula, and 5) improved 
measurability of results. Business organiza¬ 
tions that utilize online learning often assert 
a reduction in training expenses, an increase 
in productivity, and a decrease in employee 
attrition (Rich, 2001). 

Business organizations use online learning in 
a variety of ways. For example, Siemens 
uses online learning to deliver a performance 
simulation course to 10,000 employees 
world-wide (Rich, 2001). IBM employs vari¬ 
ous online learning methods in its world¬ 
wide training program involving 300,000 
employees (Johnson, 2004). Ford Motor 
Company frequently uses online learning as 
an easy way to brief its salespeople and me¬ 
chanics on the latest automotive products 
(Sessa, 2001). Few organizations (academic 
or industry-related), however, have taken 
the time and effort to assess the perceived 
effectiveness of online learning, as reported 
by learning participants. 


Variations of Online Learning 

As popular as online learning has become in 
both industry and academia, one prevailing 
question remains: what is the most effective 
utilization of online learning? (i.e., content 
delivered completely online, a hybrid model 
of content that is partially delivered online 
and partially delivered in the classroom, or 
on-ground delivery with an online supple¬ 
ment). Industry has typically favored the 
hybrid approach, opting for a combination of 
online learning and "on-ground" instruction. 
Sometimes labeled a "blended approach," 
the combination of classroom education and 
online learning has been used by industry to 
maintain educational quality and foster" . . . 
greater experiences in interpersonal rela¬ 
tionships" (Trierweiler & Rivera, 2005, p. 3). 

Academia has also taken comfort in the hy¬ 
brid approach to online learning. In their 
study involving undergraduate management 
courses, Drennan, Kennedy, and Pisarski 
found that research subjects "... consi¬ 
dered it important to find a balance between 
the use of emerging technologies and tradi¬ 
tional face-to-face lectures" (2005, p. 332). 
In a subsequent study involving undergra¬ 
duate accounting students, researchers 
found that a hybrid or blended approach to 
online learning was superior to course con¬ 
tent delivered entirely online. Specifically, 
the researchers determined that "The use of 
technology as an exclusive course delivery 
method was considered insufficient. Face-to- 
face tutorials and group learning were cen¬ 
tral to both student confidence and the 
learning process" (Flynn, Concannon, & 
Bheachain, 2005, p. 433). 

3. RESEARCH QUESTIONS 

Information Technology/ Information Sys¬ 
tems (IT/IS) curricula can present unique 
challenges to the delivery of course content 
in an online format that are not found in tra¬ 
ditional business and management pro¬ 
grams. Many courses in an IT/IS curricula 
involve instruction in computer programming 
languages, requiring hands-on development 
and extensive drill and practice while other 
courses entail theoretical concepts; both of 
which can require an increased interaction 
with IT/IS faculty. It is not yet clear if online 
learning methods are conducive to the deli¬ 
very of such course content. Furthermore, it 
is not clear as to what degree (if any) of on- 
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line learning is effective in delivering IT/IS- 
specific course content. Although research 
has been conducted with business and man¬ 
agement students in higher education to 
assess the perceived effectiveness of online 
learning, additional research needs to be 
conducted with IT/IS students. 

The present research examined the effec¬ 
tiveness of online learning, as reported by 
undergraduate, graduate, and post-graduate 
IT/IS students attending a private, medium¬ 
sized university to determine if online learn¬ 
ing is an effective means of delivery for 
IT/IS-related content. Additionally, the 
study sought to determine which delivery 
method (i.e., classroom, partially online, or 
completely online, or on-ground with online 
supplement) is perceived to be more effec¬ 
tive. 

The study also attempted to determine 
which delivery method is most appropriate 
for specific IT/IS-related subject areas. The 
IT/IS-related subject areas examined in this 
research included: 1) Software Develop¬ 
ment, 2) Network Administration and Securi¬ 
ty, 3) Web Development / Web Program¬ 
ming, 4) Multimedia / Graphics, 5) Office / 
Productivity Software, 6) Project Manage¬ 
ment, 7) Systems Analysis and Design, 8) 
Certification Courses, 9) Operating Systems, 
10) Database, and 11) Business Intelligence. 
In order to develop the 11 subject areas, 
existing classifications from ABET (Accredita¬ 
tion Board of Engineering and Technology), 
ACM (Association for Computing Machinery), 
and AIS (Association for Information Sys¬ 
tems) job categories were used as guide¬ 
lines. 

Finally, the study attempted to determine 
the typical demographic profile of students 
who prefer an online format to a traditional 
on-ground format. 

Specifically, the study was conducted to an¬ 
swer the following research questions: 

1. Which non-tradition instructional 
method of course content delivery 
(i.e., online, partially online, on¬ 
ground with online supplements) is 
most effective for IT/IS courses? 

2. For each grouping of IT/IS subject 
areas presented to students, which 
instructional method of course con¬ 


tent delivery do students perceive as 
providing the best learning ? 

3. What is the typical demographic pro¬ 
file of students who prefer an online 
format to a traditional on-ground 
format? 

4. METHODS AND PROCEDURES 

The research involved the administration of 
a survey instrument that consisted of 22 
closed-ended questions. The results from all 
22 questions were not used in this study. 
Question 1 of the survey asked the partici¬ 
pants if (given a choice for the same course) 
they would prefer an online format over an 
on-ground format. Those responding "Yes" 
to Question 1 were asked about the reasons 
for choosing an online course. Question 3 
asked the participants if they had taken an 
online course (partially or completely online) 
in the past. Those respondents who ans¬ 
wered "Yes" to Question 3 were then asked 
why they chose online and what training 
they received before enrolling in the course. 
The results from the above questions were 
not used in this study but may be used in 
future research. If the respondent answered 
"No" to Question 3, they were then branched 
to Question 10. Questions 10, 11, and 12 of 
the survey instrument asked participants to 
select the best delivery method for each of 
the eleven IT/IS subject areas. Valid res¬ 
ponses for each subject area included "On¬ 
ground Instruction," "Completely Online," 
"Hybrid/Partially Online", and "On-ground 
with Online Supplement." 

Question 13 of the survey instrument asked 
participants to report whether the University 
should "Offer More Courses," "Keep Course 
Offerings the Same," or "Offer Less Courses" 
for each of the four delivery methods. The 
results from Question 13 were not included 
in this research study. Finally, Questions 14 
through 22 solicited demographic informa¬ 
tion from the participants, such as age 
range, gender, and degree program, em¬ 
ployment status. 

The population for this survey consisted of 
498 computing majors enrolled in the Spring 
2009 semester. Of these 498 students, 248 
were at the undergraduate levels and 250 
were at the graduate and post-graduate lev¬ 
el. The undergraduate majors included a 
Bachelor of Science degree in Computer and 
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Information Systems and a Bachelor of 
Science degree in Information Sciences; 
both of which are accredited by the Accredi¬ 
tation Board of Engineering Technology - 
Computing Accreditation Commission (ABET- 
CAC). 

At the graduate and post-graduate levels, a 
total of six Master's degrees and one doctor¬ 
al degree are represented in the data popu¬ 
lation. Additionally, only 39 students (38 
undergraduate and 1 graduate) were resi¬ 
dent students. 

A total of 155 students responded to the 
survey. The students completed the online 
survey on their own time and submitted 
their anonymous results directly into an 
electronic database for analysis. The survey 
results were analyzed using SPSS statistical 
software. Statistical frequencies were used 
to answer the research questions posed in 
section 3. 

5. RESULTS 

In order to answer the first research ques¬ 
tion, (i.e., Which non-traditional instructional 
method of course content delivery is most 
effective for IT/IS courses?) the survey in¬ 
strument asked participants to select the 
"most effective" non-traditional delivery me¬ 
thod for IT/IS-related courses. The available 
delivery methods were: Completely Online, 
Partially Online, or On-Ground with Online 
Supplement. The results are summarized in 
APPENDIX A, Table 1. 

Of the three non-traditional delivery me¬ 
thods, 75.5% of the research participants 
selected Onground with Online Supplement 
as Very effective or Effective as compared to 
36.8% for Completely Online and 59.2% for 
the hybrid/ partially online formats. Fur¬ 
thermore, no respondents indicated Very 
Ineffective for either On-ground with Online 
Supplement or Hybrid/Partially Online. 

The difference in effectiveness among the 
delivery methods is further illustrated by a 
comparison of mean scores. The responses 
were designed using a Likert-like scale, with 
valid responses as "Very Effective" (value = 
6), "Effective" (value = 5), Somewhat Effec¬ 
tive" (value = 4), "Somewhat Ineffective" 
(value = 3), Ineffective" (value = 2), and 
"Very Ineffective" (value = l).The mean 
score for Completely Online = 4.07) indi¬ 
cates that the respondents, in general, con¬ 


sidered this delivery method to be "Some¬ 
what effective." The mean score for Hybrid 
/Partially Online (y = 4.65) indicates that the 
respondents, in general, considered the hy¬ 
brid delivery method to fall between "Some¬ 
what effective" and "Effective" (leaning more 
toward "Effective"). Finally, the mean score 
for On-ground with Online Supplement (% = 
5.04) indicates that the respondents consi¬ 
dered this delivery method to be "Effective." 

In order to determine any statistical signific¬ 
ance among the three instructional methods, 
a series of paired-samples T-Tests was con¬ 
ducted (at the 95% confidence level). The 
first paired-samples T-Test compared Com¬ 
pletely Online to Hybrid/Partially Online. 
The results indicated that the difference be¬ 
tween Completely Online and Hybr¬ 
id/Partially Online was statistically significant 
(p = .000). 

The second paired-samples T-Test compared 
Hybrid/Partially Online to On-ground with 
Online Supplement. The results from the 
second T-Test indicated that the difference 
between Hybrid/Partially Online and On¬ 
ground with Online Supplement was statisti¬ 
cally significant (p = .001). 

Finally, the third paired-samples T-Test 
compared Completely Online to On-ground 
with Online Supplement. The results from 
this third T-Test indicated that this differ¬ 
ence was statistically significant (p = .000). 
The results of all three paired-samples T- 
Tests are summarized in APPENDIX B, Table 
2 . 

In order to answer the second research 
question (i.e., For each grouping of IT/IS 
subject areas presented to students, which 
instructional method of course content deli¬ 
very do students perceive as providing the 
best learning ?) the survey instrument asked 
participants to select the instructional me¬ 
thod of course content delivery that provides 
the " best learning" for each grouping of 
IT/IS-related topics. The available delivery 
methods were: On-ground, Completely On¬ 
line, Partially Online, and On-Ground with 
Online Supplement. The results from these 
survey questions are summarized in 
APPENDIX C, Table 3. 

More research participants selected On¬ 
ground or Onground with Online Supplement 
as their primary course delivery method to 
provide the best learning (i.e., 9 of the 11 
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Subject Areas). Furthermore, Completely 
Online was the least selected delivery me¬ 
thod for all Subject Areas (with the excep¬ 
tion of the Office / Productivity Software 
category). 

The third research question (i.e., What is the 
typical demographic profile of students who 
prefer an online format to a traditional on¬ 
ground format?) sought to determine the 
typical demographic profile of students who 
prefer an on-line format to a traditional on¬ 
ground format? The results from these sur¬ 
vey questions are summarized in APPENDIX 
D, Table 4. 

As the results in Table 4 indicate, it is diffi¬ 
cult to use demographic factors to differen¬ 
tiate a student who prefers any online for¬ 
mat over a traditional (i.e., on-ground) for¬ 
mat. A few demographic statistics, howev¬ 
er, from Table 4 are notable. For example, 
students who classified themselves as Grad¬ 
uate and ACE (Adult and Continuing Educa¬ 
tion) seemed to prefer the online format to 
the on-ground format. In addition, those 
students who reside 6-10 miles from cam¬ 
pus also prefer the online format. 

6. CONCLUSIONS 

The present research surveyed undergra¬ 
duate, graduate, and post-graduate students 
as to which non-tradition instructional me¬ 
thod of course content delivery (i.e., online, 
partially online, on-ground with online sup¬ 
plements) is most effective for IT/IS 
courses, as well as, which instructional me¬ 
thod of course content delivery provides the 
best learning for each grouping of IT/IS sub¬ 
ject areas. Also, this study profiled groups of 
students who prefer an online format to a 
traditional on-ground format 

In relation to the non-traditional delivery 
method, overall, the research participants 
rated the perceived effectiveness of On- 
Ground with an Online Supplement or Hybr¬ 
id/Partially Online courses as higher than 
that of completely online courses. These 
findings are consistent with the findings of 
past studies in which the researchers deter¬ 
mined that students prefer a combination of 
on-ground and online instruction, rather 
than instruction delivered exclusively online 
(Drennan et al. and Flynn et al.). Therefore, 
the present study suggests that online in¬ 
struction should be coupled with traditional 


(i.e., on-ground) methods in order to pro¬ 
vide the most effective delivery of content. 

Further supporting results from the first re¬ 
search question, respondents perceived a 
combination of online and on-ground me¬ 
thods as providing the best learning. Specif¬ 
ically, respondents reported On-Ground, and 
On-Ground with an Online Supplement, and 
Hybrid/Partially Online, as formats which 
provide better learning than the completely 
online format. The only caveat to the above 
findings involved the Office/Productivity 
Software Subject Area. For Of¬ 

fice/Productivity Software topics, the res¬ 
pondents felt that Completely Online pro¬ 
vided the best learning. The Of¬ 
fice/Productivity exception may be explained 
by the fact that many of today's students 
enter college with a prior working knowledge 
of spreadsheet, word-processing, and pres¬ 
entation software. 

The analysis of demographic factors in rela¬ 
tion to a student's decision to take (or not 
take) an online course was much more diffi¬ 
cult. The current research found that very 
few demographic factors emerged as in¬ 
fluencing a student's choice of delivery me¬ 
thod. Student level and distance from cam¬ 
pus, however, did surface as possible influ¬ 
ences to the course delivery decision. Spe¬ 
cifically, the current study found that Gradu¬ 
ate students and other "non-traditional" stu¬ 
dents may prefer the online format. Fur¬ 
ther, the present study found that students 
who live between 6-10 miles from campus 
may also prefer online delivery to on¬ 
ground. 

The results of this study may seem surpris¬ 
ing when one considers the original target 
audience of distance learning programs: 
non-traditional students, who cannot attend 
traditional courses due to employment, dis¬ 
tance from campus, and family responsibili¬ 
ties. Many past studies supporting distance 
learning programs, however, did not diffe¬ 
rentiate between technical and non-technical 
degree programs. 

In contrast, the current study surveyed stu¬ 
dents enrolled in IT/IS programs. Since 
IT/IS programs involve content that is quan¬ 
titative and technically-oriented, some stu¬ 
dents may have difficulty comprehending 
such content in an online format. The tech¬ 
nical nature of such course content may ex¬ 
plain why IT/IS students in the current study 
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preferred an on-ground delivery format or 
an online format that involved an on-ground 
component. Both the On-Ground and the 
On-Ground with Online Supplement formats 
would give IT/IS students more interaction 
with faculty. This increased interaction, as 
discussed previously, may be critical in con¬ 
veying technical content. 

Although the results cannot be generalized 
to all IT/IS students in all IT/IS programs, it 
may be inferred that the IT/IS students in 
this study preferred classroom delivery of 
course content for IT/IS-related topics. 
However, if online delivery is to be incorpo¬ 
rated into an IT/IS curriculum, the current 
study suggests that online delivery should 
be provided as a hybrid (i.e., blended learn¬ 
ing, or as a supplement to traditional, on¬ 
ground delivery. 

As online learning delivery method continues 
to gain popularity, faculty and advisors must 
be equipped to help students achieve suc¬ 
cess. By recognizing variables that contri¬ 
bute to the success of online learners, edu¬ 
cators can better prepare students to choose 
between different learning formats and 
across various subject matter areas. 
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APPENDIX A 


TABLE 1: PERCEIVED EFFECTIVENESS BY DELIVERY METHOD 


Effectiveness 

Completely 

Online 

Hybrid / 
Partially Online 

On-ground with 
Online Supplement 

Count 

Percent 

Count 

Percent 

Count 

Percent 

Very Effective 

14 

9.2% 

30 

19.4% 

57 

36.7% 

Effective 

43 

27.6% 

62 

39.8% 

60 

38.8% 

Somewhat Effective 

57 

36.7% 

47 

30.6% 

27 

17.3% 

Somewhat Ineffective 

27 

17.3% 

11 

7.1% 

9 

6.1% 

Ineffective 

9 

6.1% 

5 

3.1% 

2 

1.0% 

Very Ineffective 

5 

3.1% 

0 

0.0% 

0 

0.0% 

Total 

155 

100.0% 

155 

100.0% 

155 

100.0% 

Mean 

4.07 

4.65 

5.04 

SD 

1.160 

.975 

.941 


APPENDIX B 


TABLE 2: PAIRED-SAMPLES T-TEST RESULTS 




Paired Differences 







95% Confidence 
Interval of the Dif¬ 
ference 






Mean 

Std. Devi¬ 
ation 

Std. Error 
Mean 

Lower 

Upper 

t 

Ddf 

Sig. (2- 
tailed) 

Pair 

1 

Completely Online 
- Hybrid/Partially 
Online 

-.582 

1.121 

.113 

-.806 

-.357 

-5.137 

97 

.000 

Pair 

2 

Hybrid/Partially 
Online - On- 
Ground with Online 
Supplement 

-.388 

1.172 

.118 

-.623 

-.153 

-3.275 

97 

.001 

Pair 

3 

Completely Online 
- On-Ground with 
Online Supplement 

-.969 

1.615 

.163 

-1.293 

-.646 

-5.943 

97 

.000 
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APPENDIX C 

TABLE 3: BEST LEARNING BY DELIVERY METHOD AND SUBJECT AREA 


Subject Area 

On-ground 

Completely 

Online 

Hybrid/Partially 

Online 

On-ground 
with Online 
Supplement 

Count 

% 

Count 

% 

Count 

% 

Count 

% 

Software Devel¬ 
opment / 
Programming 

53 

34.2% 

14 

9.0% 

40 

25.8% 

48 

31.0% 

Network 
Administration / 
Security 

49 

31.6% 

18 

11.6% 

36 

23.2% 

52 

33.5% 

Web Develop¬ 
ment / Web 
Programming 

37 

23.9% 

25 

16.1% 

50 

32.3% 

43 

27.5% 

Multimedia / 
Graphics 

37 

23.9% 

25 

16.1% 

43 

27.7% 

50 

32.3% 

Office / Produc¬ 
tivity Software 

33 

21.3% 

49 

31.6% 

38 

24.5% 

35 

22.6% 

Project 

Management 

44 

28.4% 

28 

18.1% 

34 

21.9% 

49 

31.6% 

System Analysis 
& Design 

48 

31.0% 

20 

12.9% 

37 

23.9% 

50 

32.3% 

Certification 
Courses (e.g., 
A+,N + ) 

50 

32.3% 

24 

15.5% 

38 

24.5% 

43 

27.7% 

Operating 

Systems 

44 

28.4% 

27 

17.4% 

41 

26.5% 

43 

27.7% 

Database 

58 

37.4% 

17 

11.0% 

36 

23.2% 

44 

28.4% 

Business Intelli¬ 
gence (e.g. Data 
Warehousing, 

Data Mining) 

45 

29.0% 

19 

12.3% 

44 

28.4% 

47 

30.3% 
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APPENDIX D 


TABLE 4: DEMOGRAPHIC PROFILE OF STUDENTS WHO PREFER/DO NOT 

PREFER ONLINE FORMAT 


Demographic 

Prefer 

Online Format 

Do NOT Prefer On¬ 
line Format 


Total 


Count 

Percentage 

Count 

Percentage 

Count 

Percentage 

Student Level 







Undergraduate 

14 

31.1% 

31 

68.9% 

45 

29.0% 

Graduate 

26 

55.3% 

21 

44.7% 

47 

30.0% 

Doctorate 

10 

23.8% 

32 

76.2% 

42 

27.1% 

ACE 

11 

64.7% 

6 

35.3% 

17 

11.0% 

Integrated 

2 

50.0% 

2 

50.0% 

4 

2.9% 

Sex 







Male 

46 

43.0% 

61 

57.0% 

107 

69.0% 

Female 

17 

35.4% 

31 

64.6% 

48 

31.0% 

Age range 







18-20 

5 

26.3% 

14 

73.7% 

19 

12.2% 

22-30 

29 

49.2% 

30 

50.8% 

59 

38.1% 

31-40 

18 

42.9% 

24 

51.1% 

42 

27.1% 

41-50 

10 

41.7% 

14 

58.3% 

24 

15.5% 

51-60 

1 

14.3% 

6 

85.7% 

7 

4.5% 

61 or older 

0 

0.00% 

4 

100.0% 

4 

2.6% 

Employment Status 







Part-Time Job 

10 

33.3% 

20 

66.7% 

30 

19.4% 

Full-Time Job 

50 

45.0% 

61 

55.0% 

111 

71.6% 

Not Employed 

3 

21.4% 

11 

78.6% 

14 

9.0% 

Enrollment Status 







Full-Time 

44 

37.9% 

72 

62.1% 

116 

75.0% 

Part-Time 

19 

48.7% 

20 

51.3% 

39 

25.0% 

Living Status 







Resident 

2 

22.2% 

7 

77.8% 

9 

5.8% 

Commuter 

61 

41.8% 

85 

58.2% 

146 

94.2% 

Previous Online Course 







Yes 

47 

48.0% 

51 

52.0% 

98 

63.0% 

No 

16 

28.1% 

41 

71.9% 

57 

37.0% 

Distance from Campus 







0 - 5 Miles 

8 

26.7% 

22 

73.3% 

30 

19.4% 

6 - 10 Miles 

15 

60.0% 

10 

40.0% 

25 

16.1% 

10 - 15 Miles 

7 

38.9% 

11 

61.1% 

18 

11.6% 

15+ Miles 

33 

40.2% 

49 

59.8% 

82 

52.9% 
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